LS-SVM Hyperparameter Selection with a Nonparametric Noise Estimator
نویسندگان
چکیده
This paper presents a new method for the selection of the two hyperparameters of Least Squares Support Vector Machine (LS-SVM) approximators with Gaussian Kernels. The two hyperparameters are the width σ of the Gaussian kernels and the regularization parameter λ. For different values of σ, a Nonparametric Noise Estimator (NNE) is introduced to estimate the variance of the noise on the outputs. The NNE allows the determination of the best λ for each given σ. A Leave-one-out methodology is then applied to select the best σ. Therefore, this method transforms the double optimization problem into a single optimization one. The method is tested on 2 problems: a toy example and the Pumadyn regression Benchmark.
منابع مشابه
Nonparametric identification of LPV models under general noise conditions: an LS-SVM based approach
Parametric identification approaches in the Linear Parameter-Varying (LPV) setting require optimal prior selection of a set of functional dependencies, used in the parametrization of the model coefficients, to provide accurate model estimates of the underlying system. Consequently, data-driven estimation of these functional dependencies has a paramount importance, especially when very limited a...
متن کاملMorozov, Ivanov and Tikhonov Regularization Based LS-SVMs
This paper contrasts three related regularization schemes for kernel machines using a least squares criterion, namely Tikhonov and Ivanov regularization and Morozov’s discrepancy principle. We derive the conditions for optimality in a least squares support vector machine context (LS-SVMs) where they differ in the role of the regularization parameter. In particular, the Ivanov and Morozov scheme...
متن کاملEstimating a Concave Distribution Function from Data Corrupted with Additive Noise
We consider two nonparametric procedures for estimating a concave distribution function based on data corrupted with additive noise generated by a bounded decreasing density on (0,∞). For the maximum likelihood (ML) estimator and least squares (LS) estimator, we state qualitative properties, prove consistency and propose a computational algorithm. For the LS estimator and its derivative, we als...
متن کاملSustainable Supplier Selection by a New Hybrid Support Vector-model based on the Cuckoo Optimization Algorithm
For assessing and selecting sustainable suppliers, this study considers a triple-bottom-line approach, including profit, people and planet, and regards business operations, environmental effects along with social responsibilities of the suppliers. Diverse metrics are acquainted with measure execution in these three issues. This study builds up a new hybrid intelligent model, namely COA-LS-SVM, ...
متن کاملThe differogram: Non-parametric noise varia estimation and its use for model selection
achines; Model-free estimates of the noise variance are important in model selection and tuning parameters. In this paper a data representation is discussed which leads to estimator suitable for multivariate data. Its visual representation—called the differogra here—is based on the 2-norm of the differences of input and output data. The crucial of locality in this representation is translated a...
متن کامل